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ABSTRACT 

In this paper, a system for facial recognition to identify miss- 
ing and found people in Hajj and Umrah is described as a web 
portal. Explicitly, we present a novel algorithm for recog- 
nition and classifications of facial images based on apply- 
ing 2DPCA to a 2D representation of the Histogram of ori- 
ented gradients (2D-H0G) which maintains the spatial rela- 
tion between pixels of the input images. This algorithm al- 
lows a compact representation of the images which reduces 
the computational complexity and the storage requirments, 
while maintaining the highest reported recognition accuracy. 
This promotes this method for usage with very large datasets. 
Large dataset was collected for people in Hajj. Experimental 
results employing ORL, UMIST, JAFFE, and HAJJ datasets 
confirm these excellent properties. 

Index Terms — Facial recognition, 2DPCA, 2D-H0G 

1. INTRODUCTION 

Human facial recognition seems at the first glance to be a 
challenging task due to several reasons such as changes in 
haircuts, beards, wearing glasses etc. Also human faces de- 
pend on people's race and culture. On the other hand, more 
than ten million people visits KSA yearly, among them sev- 
eral thousand got missing due to the huge crowds in the two 
holy cities of Makkah and Madinah. 

We propose a Crowd-Sensing system as shown in Fig. [T] 
that is composed of a web portal including a facial recognition 
algorithm. The Crowd-Sensing system is established to sup- 
port the existing efforts to manage the crowds and solve the 
missing and found problem during Hajj and Umrah seasons in 
KSA. The goal of this Crowd-Sensing system is to use tech- 
niques from computer vision and image processing to develop 
a portal website for Hajj missing and found people ifH. ll4ll. 

Many algorithm based on principal component analysis 
(PCA) have been applied to the facial recognition problem. 
The main idea of PCA is to find the vectors that best ac- 
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Fig. 1. Crowd-Sensing system. The system consists of data 
collectors (mobile, cameras, PCs), main server, search engine, 
and an alerting system. 



count for the distribution of facial images within the entire im- 
age space. In 1991 Turk and Pentland (Tl, (61 developed the 
Eigenfaces method based on the principal component anal- 
ysis (PCA). Recently Yang et. al. |8| proposed the two di- 
mensional PCA (2DPCA) technique, which has many advan- 
tages over the PCA method. It is simpler for image fea- 
ture extraction, better in recognition rate and more efficient 
in computation complexity. In this contribution we are pre- 
senting an efficient method based on applying 2DPCA to a 
proposed 2D representation of the HOG (2D-H0G) where it 
maintains the spatial relation between pixels. In addition, this 
algorithm allows compact representation of the images which 
yields excellent recognition speed and storage requirements, 
while maintaining the highest reported recognition accuracy. 

2. CROWD-SENSING SYSTEM DESCRIPTION 

In this section, we describe the Crowd-Sensing system for 
recognizing missing and found people during Hajj and Umrah 
seasons in KSA. The main idea is that pilgrims send images 
of their missing/found individuals and the developed web por- 




tal as shown in Fig. [21 can detect and recognize their faces. 
Hence then, all person information can be known easily. The 
Crowd-Sensing tool can be used for road management, me- 
dia, surveillance, crowd estimation, crowd flow monitoring, 
crowd data collection (speed, size, direction, and stampede) 
and flow jams reduction. 

The proposed system has several advantages including a 
tool for identifying missing and found people during dense 
crowded situations. It also contains a search engine for all 
missing objects. Furthermore, it contains a database, which 
can save up to three millions records. The system works as 
follows: 

i) A database will be setup for all people observing Hajj 
and Umrah. 

ii) A person photo will be uploaded to the website. A 
computer program will be used to recognize the miss- 
ing/found person employing the novel facial recognition 
algorithm presented in this paper. 

iii) The system will be capable of recognizing human facial 
images in the presence of various anomalies. 

iv) It worth to mention that. In addition to facial recogni- 
tion application presented in this paper the system will 
be able to detect, count recognize crowds and people be- 
havior. Our method will be robust, and not subject to 
failure against pollution, temperatures, or any other at- 
mosphere changes. 

3. PROPOSED ALGORITHM (2DHOG /2DPCA) 

In 2006 Dalai and Triggs ^ introduced a single window hu- 
man detection algorithm with excellent detection results. This 
method uses a dense grid of Histograms of Oriented Gradients 
(HOG) for feature extraction and using linear Support Vector 
Machine (SVM) for classification. The HOG representation 
has several advantages. It captures edge or gradient structure 
that is the main feature of local shape, and it is robust for 



Fig. 3. Face Recognition System Description. Figures (a) 
and (b) show steps of the facial training and testing detection 
algorithms. 



illumination changes, invariant for human clothes, and back- 
ground changes. 

In this paper, a novel algorithm for human face recogni- 
tion is introduced, where 2DPCA is applied to Histogram of 
Oriented Gradients (HOG) represented into n orientation bins 
that are layers in 2D format. Feature extraction process can 
be either in spatial or transform domains ll2ll. 131. 

The technical contributions of this part are two folds, first 
representing the HOG features in 2D format so that the rela- 
tion between HOG features is maximized. Second, each bin 
will contribute to the decision separately in a parallel struc- 
ture which reduces the computational time while achieving 
excellent accuracy levels, as other state-of-the-art approaches. 
The 2D-HoG B-bin features extraction, consists of a parallel 
structure (layers) as shown in Fig.O 

The B bins, where each bin represents one of the desired 
angles, are arranged in B matrices, where spatial relations 
are maintained. Each bin will be dealt with separately in a 
parallel structure. The stored feature vector representing the 
training image will be formed by the concatenation of fea- 
tures extracted from all B bins. In the testing mode, a mini- 
mum distance classifier combined with a voting scheme will 
be used to measure the similarity between the features repre- 
senting testing and training images. 

ORL Dataset: The ORL dataset consists of 400 images of 
40 different individuals (10 images for each persons), where 
pose and facial expressions are varying, Fig.[7t. 
UMIST Dataset: The UMIST dataset consists of 564 images 
of 20 individuals (mixed race/gender/appearance). Each in- 
dividual is shown in a range of poses from profile to frontal 
views Fig.[7]3. 

JAFFE Dataset: The JAFFE database consists of 10 persons. 




Fig. 4. Flowchart for the training mode 
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Fig. 5. Flowchart for the testing mode 



Fig. 6. An example for the proposed 2DHoG Algorithm 



The images per person vary from 20 to 23 , where facial ex- 
pressions are varying, Fig. [7]:. 

Hajj Dataset: HAJJ dataset was taken randomly during Hajj 
and Umrah seasons. It consists of 100 different individuals 
(images per person vary from 3 to 7), where pose, facial ex- 
pressions, and appearance (shaved hair, wearing glasses, etc.) 
are varying. Samples for these datasets are shown in Fig.[7]l. 

4. EXPERIMENTAL RESULTS AND ANALYSIS 

Three experiments were conducted on the ORL, UMIST, and 
JAFFE dataset. In the first experiment, five images were used 
for training and the remaining images were used for testing. 
In the second experiment three images were used for training 
and the remaining images were used for testing. In the third 
experiment the leave-one-out strategy was used. Experimen- 
tal results were compared with those obtained by PCA and 
2DPCA methods. 

In our experiments, the recognition system works as fol- 
lows. The image of size 112x96 in the spatial domain or of 
size 56x48 in the Discrete Wavelet Transform (DWT) domain 
was used. The cell size is 4x4 pixels, and the block size is 2x2 
cells, with no overlapping. The ten eigenvectors correspond- 
ing to the ten largest eigenvalues of the covariance matrix 
were obtained. Consistent accuracies were obtained in both 
the spatial and transform domain. The storage and compu- 
tational requirements were reduced in the transform (DWT) 
domain. Results shown in the following tables were obtained 
employing DWT. 

As shown in Table [H the recognition accuracy has im- 
proved compared to the excellent recognition accuracy of the 
2DPCA method. From Table [2l the computational complex- 
ity is less than those required by PCA, and 2DPCA methods. 
Tabled shows comparable storage requirements to those re- 
quired by 2DPCA. method. 
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Fig. 7. Samples of facial images (a) ORL, (b) UMIST, (c) 
JAFFE, (d) HAJJ 



In addition, two experiments were conducted on the Hajj 
dataset with the same parameters used in the previous experi- 
ments. In the first experiment, two images were used for train- 
ing and one image was used for testing for each individual. It 
worth to note that these images are with different orientations 
and appearance. In the second experiment, one frontal im- 
age was used for training, another frontal image with different 
appearance and scale was used for testing. In the first experi- 
ment, an accuracy of 92.5% was obtained, while in the second 
experiment, 100% accuracy was achieved. These excellent 
properties promote our proposed method for usage with large 
scale datasets for real time applications. 



Table 1. Recognition accuracy for experiment I employing 
PCA, 2DPCA, and 2D-HOG/2DPCA methods. 





PCA 


2DPCA 


2D-HOG/2DPCA 


ORL 
UMIST 
JAPANESE 


92.5% 
85% 
93.5% 


96% 
88% 
96.00% 


97% 
90.35% 
100.00% 



Table 2. Computational Complexity (CPU 2.3 GHz, 
RAM IG) for experiment I on ORL dataset employing PCA, 
2DPCA, and 2DHOG/2DPCA. 





PCA 


2DPCA 


2DHOG/2DPCA 


Training 


^Mode 


lOlsec 


14sec 


10.4 sec 


Testing 


Mode 


2.1 sec 


1.5 sec 


0.82 sec 



5. CONCLUSION 

In this contribution, a power intelligent image processing sys- 
tem applied to recognition and classification of facial images 



Table 3. Storage requirements for experiment I employing 
2DPCA, and 2DHOG/2DPCA. 





ORL 


UMIST 


JAFFE 


2DPCA 
2DHOG/2DPCA 


112x12 
14x12x8 


112x12 
14x12x20 


112x12 
14x12x11 



Table 4. Comparison between Recognition accuracy for ex- 
periments IJI^andJIIemplo^^ method. 





EX I 


EX II 


EX III 


ORL 
UMIST 
JAFFE 


97% 
90.35% 
100% 


84.64% 
85.18% 
100% 


100% 
100% 
100% 



is presented. Experimental results using the ORL, UMIST, 
JAFFE, and a new collected dataset for people in Hajj confirm 
the excellent performance of the new techniques presented 
in terms of recognition accuracy, speed, and storage require- 
ments. 
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